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□ furnished subsequently to this Authority in written form. 

□ furnished subsequently to this Authority in computer readable form. 

□ The statement that the subsequently furnished written sequence listing does not go beyond the disclosure in 
the international application as filed has been furnished. 

□ The statement that the information recorded in computer readable form is identical to the written sequence 
listing has been furnished. 
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□ the description, pages: 

□ the claims, Nos.: 
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□ the drawings, sheets: 

5. □ This report has been established as if (some of) the amendments had not been made, since they have been 
considered to go beyond the disclosure as filed (Rule 70.2(c)): 

(Any replacement sheet containing such amendments must be referred to under item 1 and annexed to this 
report.) 
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V. Reasoned statement under Article 35(2) with regard to novelty, inventive step or industrial applicability; 
citations and explanations supporting such statement 

1. Statement 

Novelty (N) Yes: Claims 2 

No: Claims 1,3,4 

inventive step (IS) Yes: Claims 

No: Claims 2,3 

Industrial applicability (IA) Yes: Claims 1-4 

No: Claims 



2. Citations and explanations 
see separate sheet 



VII. Certain defects in the international application 

The following defects in the form or contents of the international application have been noted: 
see separate sheet 
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1 . Reference is made to the following documents: 

D1 : WO 97/1 6799 A (Baron Ehud et al) 9 May 1 997 
D2: US-A-4 736 445 (Gundersen Steven C) 5 April 1988 
D3: WO 97/44758 A (Apple Computer) 27 November 1997 

2. Item V: Reasoned statement under Article 35(2) with regard to novelty, 
inventive step or industrial applicability; citations and explanations 
supporting such statement 

2.1 The present application fails to meet the requirements of Article 33(2) PCT 

because the subject matter of claims 1 and 4 is not novel, the reasons being as 
follows: 

As to claim 1 : 
D2 discloses: 

- A handwriting recognition system (see column 3, lines 30-31) comprising: 

- means responsive to input analogue signals representative of movement of a 
hand held writing device (see column 4, lines 8-19); 

- sampling means to provide signals representative of the acceleration of the 
writing device in at least x axis and y axis channels at a predetermined capture 
sampling rate (see column 4, lines 8-19); and 

- filtering means to remove DC level components and to provide smoothing of the 
output whereby signals representative of the movement of the pen over a period 
are supplied to a classifier for comparison with a template representative of 
characters formed (see column 4, line 60 - column 5, line 22 supported by column 
5, lines 50-64). 

This is the wording of present claim 1 , the subject-matter of which consequently is 
not novel (Article 33(2) PCT). 
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As to claim 4: 
D2 discloses: 

- A method of analysing signals from a moving hand held device (see column 3, 
lines 30-31 supported by column 4, lines 8-1 1 and column 1 , lines 6-1 1 ), the 
method comprising: 

- sampling signals at a predetermined rate (see column 4, lines 8-19), 

- passing signals through a bandpass filter to remove DC level and excess 
acceleration components (see column 4, line 60 - column 5, line 22), 

- sampling the filtered output to provide a series of vectors representing the 
position of the hand held device at periodic intervals (see column 5, lines 4-6 
supported by column 5, lines 31-34) and 

- using a classifier to compare the sample sets with predetermined templates to 
determine the character for output (see column 5, lines 50-64). 

This is the wording of present claim 4, the subject-matter of which consequently 
fails to meet the requirements of Article 33(2) PCT for lack of novelty. 



2.2 Dependent claims 2 and 3 do not contain any features which, in combination with 
the features of any claim to which they refer, meet the requirements of the PCT 
with respect to novelty and inventive step, the reasons being as follows: 

As to claim 2: 

D2 does not disclose that the classifier uses a hidden Markov model for 
comparison purposes. This, however, can be found in document D3, which the 
skilled person will surely be aware pf since both documents D2 and D3 belong to 
field of handwriting recognition. D3 discloses a recognition system for hand-written 
ideographic characters which consist of a number of subcharacters using hidden 
Markov models. By combining D2 and D3 the skilled person will consequently 
arrive at claim 2 without the exercise of an inventive step. 
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As to claim 3: 

D2 further discloses: 

the sampling means, filtering means and classifier are implemented in a digital 
computer environment (see column 4, lines 24-31 supported by figure 1). 

This is the wording of present claim 3, the subject matter of which consequently is 
not novel (Article 33(2) PCT) and does not involve an inventive step in the sense 
of Article 33(3) PCT when regarded in combination with present claim 2. 

3. Item VII: Certain defects in the international application 

The features of the claims are not provided with reference signs placed in 
parentheses (Rule 6.2(b) PCT). 

The description is not in conformity with the claims as required by Rule 5.1(a)(iii) 
PCT. 

Contrary to the requirements of Rule 5.1(a)(ii) PCT, the relevant background art 
disclosed in the documents D1 , D2 and D3 is not mentioned in the description, nor 
are these documents identified therein. 
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WO 00/57349 PCT/GBOO/01052 



HANDWRITING RECOGNITION SYSTEM 



The present invention relates to a handwriting recognition system and more 
particularly to a hardware and an algorithm for implementing such a system. 
5 In PCT application number GB98/0316 (Publication No. W099/22338) there 

is disclosed a portable computer in the form of a pen-type casing. Incorporated 
within the casing is at least one acceierometer which is used to detect movement of 
the pen with respect to its environment. By using the instrument for handwriting it is 
possible to effect data entry or transmission of signals reflecting movement, the user 

10 using either a pen tip mounted switch or a finger operated switch to indicate that 
movement is effecting a written input. 

There are many other pen-type input devices on the market in addition to 
stylus scroll pallets typically used in so-called palm top computers where handwriting 
recognition has been used. Such devices often require very precise movement which 

1 5 may not reflect natural handwriting movements for the user. One of the problems 
which makes characteristic handwriting recognition difficult is that while the 
underlying movement made by an individual to represent a particular letter may be 
consistent an element reflecting user movements due to stress and other factors will 
be present. 

20 According to the present invention there is provided a handwriting 

recognition system comprising means responsive to input analogue signals 
representative of movement of a handheld writing device, sampling means to provide 
signals representative of the acceleration of the writing device in at least x axis and y 
axis channels at a predetermined capture sampling rate, and filtering means to 

25 remove dc level components and to provide smoothing of the output whereby signals 
representative of movement of the pen over a period are supplied to a classifier for 
comparison with a template representative of characters formed. 

The classifier may use hidden Markov modelling (HMM) techniques using a 
large number of states to determine the character defined by movement. The system 

30 may include an input indicative of a user's intention that the movement is 
representative of character writing. 

According to a feature of the present there is provided a method of analysing 
signals from a moving handheld device, the method comprising sampling signals at a 
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predetermined rate, passing signals through a bandpass filter to remove dc level and 
excess acceleration components, sampling the filtered output to provide a series of 
vectors representing the position of the handheld device at periodic intervals and 
using a classifier to compare the sample sets with predetermined templates to 
5 determine the character for output. 

A handwriting recognition system in according with the invention will now be 
described by way of example only with reference to the accompanying drawing of 
which: 

Figure 1 is a block schematic diagram of the system; 
10 Figure 2 shows relative positioning of the x and y axis of the handwriting 

device of Figure 1 ; 

Figure 3 is a schematic diagram of the handheld writing device of Figure 1 in 
a particular position; 

Figure 4 shows relative input and output vector streams from the system of 
1 5 Figure 1 ; and 

Figures 5 to 9 show comparative templates for a number of different letters. 
Referring first to Figure 1 , an input device 1 such as a stylus produces x and 
y vector streams 2 and 3 which are fed into a sampling unit 4. The outputs x and y 
are generally from accelerometers or other position sensing devices within the stylus 
20 1 . Also feeding the sampling unit 4 is an output 5 from a switch indicated here as 
being in the nib section of the stylus 1 such that contact between the switch 6 and a 
surface is indicative of the stylus being used in a writing mode. It will be appreciated 
that the nib switch 6 when incorporated in a non-surface contacting stylus, such as 
that disclosed in the previously referred PCT application, may be replaced by a user 
25 operable switch. 

The output of the sampling unit 4, which samples the incoming streams at 
60 Hz for example, is passed to a bandpass filter arrangement 7 and thence to a 
down sampling unit 8 which produces digitised vectors x and y over a period of time. 
The x and y vectors are passed to a classifier 9, which uses a hidden Markov model 
30 to carry out a comparison between the vectors and templates representing written 
characters. The classifier 9 may be arranged to output to a visual display 10. 

More specifically, the stylus 1 for example comprises a simple plastic casing 
containing the electronics for transferring information to a PC. Two accelerometers 
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mounted in the stylus, for example, are used to produced the x and y outputs. The 
nib switch is a simple on/off switch connected to determine when pressure is being 
applied to the pen nib and can therefore detect when a pen for stylus 1 is writing. 

Turning briefly to Figure 2, the two accelerometers mounted in the top of the 
5 pen measure acceleration across their plane such that effectively they measure 
acceleration along the x axis 11 and the y axis 1 2 of the writing surface 14. The 
acceleration measured by the sensors is made up of two components, acceleration 
due to gravity and the acceleration as a result of stylus movement. It will be 
appreciated that the acceleration due to gravity is always present, such that when 
10 the pen is exactly horizontal both sensors would measure acceleration of 1gr. As the 
angle of the pen to the horizontal changes (as shown for example in Figure 3), the 
accelerometers are subject to Sin 0x1p where 6 is the inclination angle of the stylus 
1. 

The other component, acceleration as a result of the pen moving is produced 
15 by the acceleration and deceleration effect as the user writes. 

The acceleration of the two sensors x tota , and y tota , can be expressed as 

y total = Yg y movement Q^d X tota | X^ + ^movement 

20 The remaining items of Figure 1 are incorporated in a computer unit, for example a 
PC, and three signals, as previously indicated, 2, 3 and 5 being the two acceleration 
signals and binary signal from the pen switch are provided to the PC. 

In one embodiment the two acceleration signals are read into a normal PC 
using an RS232 port and the binary switch signal by means of the games port of a 

25 sound card. 

The sampling section must sample sufficiently regularly to capture the 
movement of the pen but should not over-sample, which would result in a waste of 
processing and storage within the PC. It has been found satisfactory for the 
purposes of the current invention to sample at a rate of 60 Hz. The acceleration 
30 signals for each channel are read in as two byte words giving a dynamic range for 
each acceleration signal from 0 to 65535. The pen nib switch is similarly sampled at 
the same rate. 
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As previously mentioned, the accelerometer signals are partly dependent on a 
component of the earth's gravitation field passing through the accelerometer of the 
stylus 1 . This results in an almost constant dc level present on the output 
corresponding to the average pen angle G while writing. The bandpass filter 7 is thus 
5 arranged to filter the signals from the two accelerometers to remove the offsets. 
Additionally, the bandpass filter smoothes the output from the sensors thus 
correcting for instability introduced by the user so that the smoothed output from the 
sensors increases robustness and facilitates matching between the x and y vectors 
and stored templates. 

10 Turning now to Figure 4, for each sample received on the PC from the 

accelerometers, the pen nib switch (or manually operable switch) indicates whether 
the stylus 1 was being used in a writing mode or not. The down sampling process 8 
uses the information to down sample acceleration samples and to retain only those 
when it was known that the stylus 1 was writing. Thus, consider Figure 4, assuming 

1 5 that the x axis and y axis vectors are as shown at 1 5 and 1 6, then the down 
sampling vectors need only be taken into account when the nib switch signal 
indicator 17 is high. This will reduce the number of samples significantly so that the 
output from the down sampling process is a time series of two dimensional vectors x 
and y as indicated at 18 and 19. 

20 Having completed processing of the acceleration measurement from the 

stylus 1 the vector stream is passed to the classifier stage 9 which takes in a series 
of vectors representing the acceleration measurements made within a given word. 
These are then compared to a set of templates which cover the range of words within 
the system vocabulary and the word which matches most closely with the unknown 

25 input word is deemed the recognised word. In this system the classifier is a hidden 
Markov model. Such models have been widely used in speech recognition and using 
a large number of states in the hidden Markov model will give the best performance 
for corresponding handwriting recognition. 

The display 10, which displays the output from the PC allows display of a 

30 word, for example, on a screen. 

While the above handwriting recognition system is intended for use with a 
series of known words which, depending on the system vocabulary entered into the 
PC, may be a large number, it will be possible to use the same kind of system to 
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validate single character entry. Using single character recognition and using cursive 
entry it is still possible to build individual words which may not be present in the 
vocabulary. There may be a lower level of confidence in words created rather than 
template determined. However, over time, the vocabulary may be expanded where 
5 multiple entries of the same word have occurred such that higher confidence levels 
may be achieved. 

A typical single accelerometer output can be seen respectively for the letters 
c, b, f and h in Figures 5 to 8. In each case the template developed here shows three 
entries on a single accelerometer for each of the letters. 

10 In Figure 9, f, b, h and c are shown in comparison so that a suitable template 

for comparison may be derived. It will be appreciated that the combination of an x 
accelerometer trace and a y accelerometer trace will serve further to emphasise the 
difference between each input letter. 

Bandpass filtering in digital form to remove dc components and high 

1 5 frequency components increases the reliability of the recognition process and 
therefore the reliability of the interpretation of the stylus input 1 . It will be 
appreciated that where the stylus 1 carries other components, for example an internal 
processing arrangement, some of the functions may be transferred from the PC to the 
stylus 1 . All of the components of sampling, bandpass filtering, down sampling and 

20 classifying can be implemented in a suitable computer program. 
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CLAIMS 

1. A handwriting recognition system comprising: 

means responsive to input analogue signals representative of movement of a 
5 handheld writing device; 

sampling means to provide signals representative of the acceleration of the 
writing device in at least x axis and y axis channels at a predetermined capture 
sampling rate; and 

filtering means to remove dc level components and to provide smoothing of 
10 the output whereby signals representative of movement of the pen over a period are 
supplied to a classifier for comparison with a template representative of characters 
formed. 

2. A handwriting recognition system as claimed in claim 1 , in which the 
1 5 classifier uses a hidden Markov model for comparison purposes. 

3. A handwriting recognition system as claimed in claim 1 or claim 2, in which 
the sampling means, filtering means and classifier are implemented in a digital 
computer environment. 

20 

4. A method of analysing signals from a moving handheld device, the method 
comprising sampling signals at a predetermined rate, passing signals through a 
bandpass filter to remove dc level and excess acceleration components, sampling the 
filtered output to provide a series of vectors representing the position of the handheld 

25 device at periodic intervals and using a classifier to compare the sample sets with 
predetermined templates to determine the character for output. 



WO 00/57349 



PCT/GB00/01052 




SUBSTITUTE SHEET (RULE 26) 



WO 00/57349 



PCT/GB0O/O10S2 




SUBSTITUTE SHEET (RULE 26) 



WO 00/57349 PCT/GBOO/01052 



3/5 




SUBSTITUTE SHEET (RULE 26) 



# • 



WO 00/57349 PCT/GB00/01052 

4/5 





SUBSTITUTE SHEET (RULE 26) 



WO 00/57349 



PCT/GB00/01052 



5/5 



Fig.9. 




26 51 76 101 126 151 176 201 226 251 




26 51 76 101 126 151 176 201 226 251 




26 51 76 101 126 151 176 201 226 251 




26 51 76 101 126 151 176 201 226 251 



SUBSTITUTE SHEET (RULE 26) 



INTERNA 



SEARCH REPORT 



Inter. ^B^Appllcatton No 

PCT/GB 00/01052 



A. CLASSIFICATION OF SUBJECT MATTER 

IPC 7 G06K9/22 



According to International Patent Classification (IPC) or to both national classification and IPC 



B. FIELDS SEARCHED 



Minimum documentation searched (classification system followed by classification symbols) 

IPC 7 G06K 



Documentation searched other than minimum documentation to the extent that such documents are included in the fields searched 



Electronic data base consulted during the international search (name of data base and. where practical, search terms used) 

EPO-Internal , PAJ, INSPEC 



C. DOCUMENTS CONSIDERED TO BE RELEVANT 



Category * 



Citation of document, with indication* where appropriate, of the relevant passages 



Reievant to claim No, 



WO 97 16799 A (GENOSSAR OMRY ; BARON EHUD 
(ID; BARON TECH LTD (ID) 
9 May 1997 (1997-05-09) 
abstract; figure 8 

* page 15, last paragraph * 

* page 22, paragraph "STEP 330" * 

US 4 736 445 A (GUNDERSEN STEVEN C) 

5 April 1988 (1988-04-05) 

column 41, line 60 -column 5, line 22 

WO 97 44758 A (INST OF SYSTEMS SCIENCE 

.APPLE COMPUTER (US)) 

27 November 1997 (1997-11-27) 

abstract 



1-4 



1-4 



□ 



Further documents are listed in the continuation of box C. 



0 



Patent family members are listed in annex. 



* Special categories of cited documents : 

"A" document defining the general state of the art which is not 

considered to be of particular relevance 
"E" earlier document but published on or after the international 

filing date 

"L" document which may throw doubts on priority claim(s) or 
which is cited to establish the publication date of another 
citation or other special reason (as specified) 

"0" document referring to an oral disclosure, use. exhibition or 
other means 

"P" document published prior to the international filing date but 
later than the priority date claimed 



"T" later document published after the international filing date 
or priority date and not in conflict with the application but 
cited to understand the principle or theory underlying the 
invention 

"X" document of particular relevance; the claimed invention 
cannot be considered novel or cannot be considered to 
involve an inventive step when the document is taken alone 

"V document of particular relevance; the claimed invention 

cannot be considered to involve an inventive step when the 
document is combined with one or more other such docu- 
ments, such combination being obvious to a person skilled 
in the art. 

document member of the same patent family 



Date of the actual completion of the international search 



28 June 2000 



Date of mailing of the international search report 



05/07/2000 



Name and mailing address of the ISA 

European Patent Office, P.B. 5818 Patentlaan 2 
NL - 2280 HV Rijswijk 
Tel. (+31-70) 340-2040, Tx. 31 651 epo nl. 
Fax: (+31-70) 340-3016 



Authorized officer 



Sonius, M 



Form PCT/ISA/210 (second sheet) (July 1992) 



INTERNAT^^AL SEARCH REPORT 

Information on patent family members 



Inter. ^ Application No 

PCT/GB 00/01052 



Patent document 
cited in search report 



Publication 
date 



Patent family 
member(s) 



Publication 
date 



WO 9716799 



09-05-1997 



AU 



7330596 A 



22-05-1997 



us 


4736445 


A 


05-04-1988 


JP 


62172478 


A 


29-07-1987 


wo 


9744758 


A 


27-11-1997 


AU 


3211897 


A 


09-12-1997 










CA 


2252370 


A 


27-11-1997 



Form PCT/ISA/210 {patent family annex) (July 1992) 



PCT/G BOO/01 052 



P/ iNT COOPERATION TREA - 



From the INTERNATIONAL BUREAU 



PCT 

NOTIFICATION OF ELECTION 

(PCT Rule 61.2) 


To: 

Commissioner 

US Department of Commerce 

1 InitoH Qtaf-oc Patent snri TrflHpmarlf 
UMUcU oidLco rdlcMl allU l laUciMaiK. 

Office, PCT 

201 1 South Clark Place Room 
CP2/5C24 

Arlington, VA 22202 
ETATS-UNIS D'AMERIQUE 

in its capacity as elected Office 


Date of mailing (day/month/year) 

22 November 2000 (22.11.00) 




International application No. 
PCT/GB00/01052 


Applicant's or agent's file reference 
A25731 WO 


International filing date (day/month/year) 
21 March 2000(21.03.00) 


Priority date (day/month/year) 
24 March 1999 (24.03.99) 


Applicant 

MILNER, Benjamin, Peter 



1. The designated Office is hereby notified of its election made: 



in the demand filed with the International Preliminary Examining Authority on: 
18 September 2000 (18.09.00) 

| | in a notice effecting later election filed with the International Bureau on: 



2. The election | X| was 

□ 



was not 



made before the expiration of 1 9 months from the priority date or, where Rule 32 applies, within the time limit under 
Rule 32.2(b). 



< 



o 





The International Bureau of WIPO 
34, chemin des Colombettes 
1211 Geneva 20, Switzerland 

Facsimile No.: (41-22) 740.14.35 


Authorized officer 

Zakaria EL KHODARY 
Telephone No.: (41-22) 338.83.38 



Form PCT/IB/331 (July 1992) 



GB0001052 



